Signal detection and visualization using point-in-time architecture databases

ABSTRACT

Systems and methods are provided for using point-in-time architecture (PTA) databases. An exemplary method includes: entering first data, received from a first data source, into a first PTA database; receiving a first instruction to process the first data using a first statistical operation; executing the first statistical operation for the first data, resulting in first output data; filtering the first output data based on a user-selected attribute; and performing multiple stages of a data processing operation for the first output data.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to and is a continuation of U.S. patentapplication Ser. No. 17/069,751, filed Oct. 13, 2020, the disclosure ofwhich is incorporated by reference in its entirety for all purposes.

TECHNICAL FIELD

The present disclosure relates to methods and systems for detecting orobtaining signals and managing, analyzing, and visualizing detectedsignals.

BACKGROUND

Researchers, scientists, industry players, academics, governmentregulators, and other stakeholders are increasingly in need of data thatis accessible, easy to interpret, analyze, and/or visualize.

SUMMARY

According to one aspect of the subject matter described in thisdisclosure, a method for using a point-in-time architecture (PTA)database is provided. The method includes receiving, using one or morecomputing device processors, first data from a first data source. Thefirst data includes a first record associated with a first attribute anda first event, and a second record associated with a second attributeand a second event. Also, the method includes organizing, using the oneor more computing device processors, the first data in a first PTAdatabase. For each record, the first PTA database includes a first timeassociated with creation of an entry in the first PTA database for therecord and a second time associated with manipulation of the entry inthe first PTA database for the record. The method includes enabling,using the one or more computing device processors, access to ahistorical version of the first PTA database without destroying acurrent version of the first PTA database.

In addition, the method includes enabling, using the one or morecomputing device processors, access to a history of manipulation of thefirst PTA database. The second data from a second data source isreceived, and the second data includes a third record associated with athird attribute and a third event, and a fourth record associated with afourth attribute and a fourth event. Moreover, the method includesorganizing, using the one or more computing device processors, thesecond data in the first PTA database or a second PTA database. A firstinstruction to process the first data and the second data using a firststatistical operation is received. The first statistical operationcomprises one or more first parameters. Also, the method includesexecuting, using the one or more computing device processors, the firststatistical operation for the first data and the second data.Additionally, the method includes receiving, using the one or morecomputing device processors, a second instruction to process the firstdata and the second data using a second statistical operation. Thesecond statistical operation comprises one or more second parameters.

Also, the method includes executing, using the one or more computingdevice processors, the second statistical operation for the first dataand the second data. The method further includes receiving, using theone or more computing device processors, a third instruction to filterfirst output data resulting from the execution of the first statisticaloperation for the first data and the second data, or filter secondoutput data resulting from the execution of the second statisticaloperation for the first data and the second data. The filtering of thefirst output data or the second output data is based on a user-selectedattribute associated with records comprised in the first data or thesecond data, respectively. Furthermore, the method includes enabling,using the one or more computing device processors, simultaneousgraphical display of the first output data resulting from the executionof the first statistical operation for the first data and the seconddata, and graphical display of the second output data resulting from theexecution of the second statistical operation for the first data and thesecond data, on a single user interface.

According to another aspect of the subject matter described in thisdisclosure, a system for using a point-in-time architecture (PTA)database, the system is provided. The system includes at least one PTAdatabase having a first data set and a second data set. Also, the systemincludes one or more computing device processors. The system furtherincludes one or more computing device memories, coupled to the firstdata set, the second data set, and the one or more computing deviceprocessors. The one or more computing device memories store instructionsexecuted by the one or more computing device processors.

The instructions are configured to: receive first data from a first datasource, wherein the first data comprises a first record associated witha first attribute and a first event, and a second record associated witha second attribute and a second event; organize the first data in afirst PTA database, wherein for each record, the first PTA databasecomprises a first time associated with creation of an entry in the firstPTA database for the record and a second time associated withmanipulation of the entry in the first PTA database for the record;enable access to a historical version of the first PTA database withoutdestroying a current version of the first PTA database; enable access toa history of manipulation of the first PTA database; receive second datafrom a second data source, wherein the second data comprises a thirdrecord associated with a third attribute and a third event, and a fourthrecord associated with a fourth attribute and a fourth event; organizethe second data in the first PTA database or a second PTA database;receive a first instruction to process the first data and the seconddata using a first statistical operation, wherein the first statisticaloperation comprises one or more first parameters; execute the firststatistical operation for the first data and the second data; receive asecond instruction to process the first data and the second data using asecond statistical operation, wherein the second statistical operationcomprises one or more second parameters; execute the second statisticaloperation for the first data and the second data; receive a thirdinstruction to filter first output data resulting from the execution ofthe first statistical operation for the first data and the second data,or filter second output data resulting from the execution of the secondstatistical operation for the first data and the second data, whereinthe filtering of the first output data or the second output data isbased on a user-selected attribute associated with records comprised inthe first data or the second data, respectively; and enable simultaneousgraphical display of the first output data resulting from the executionof the first statistical operation for the first data and the seconddata, and graphical display of the second output data resulting from theexecution of the second statistical operation for the first data and thesecond data, on a single user interface.

According to another aspect of the subject matter described in thisdisclosure, a method for using a point-in-time architecture (PTA)database is provided. The method includes receiving, using one or morecomputing device processors, first data from at least a first datasource. The first data comprises a first record associated with a firstattribute and a first event, and a second record associated with asecond attribute and a second event. Also, the method includesorganizing, using the one or more computing device processors, the firstdata in a first PTA database. For each record, the first PTA databaseincludes a first time associated with creation of an entry in the firstPTA database for the record and a second time associated withmanipulation of the entry in the first PTA database for the record.Moreover, the method includes enabling, using the one or more computingdevice processors, access to a historical version of the first PTAdatabase without destroying a current version of the first PTA database.The method further includes enabling, using the one or more computingdevice processors, access to a history of manipulation of the first PTAdatabase.

In addition, the method includes receiving, using the one or morecomputing device processors, a first instruction to process the firstdata using at least one first statistical operation. The at least onefirst statistical operation includes one or more first parameters. Themethod further includes executing, using the one or more computingdevice processors, the at least one first statistical operation for thefirst data. Additionally, the method includes receiving, using the oneor more computing device processors, a second instruction to filterfirst output data resulting from the execution of the at least one firststatistical operation for the first data. The filtering of the firstoutput data or the second output data is based on a user-selectedattribute associated with records comprised in the first data,respectively. Furthermore, the method includes enabling, using the oneor more computing device processors, graphical display of the firstoutput data resulting from the execution of the at least one firststatistical operation for the first data.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure is illustrated by way of example, and not by way oflimitation in the figures of the accompanying drawings in which likereference numerals are used to refer to similar elements. The variouselements shown in the figures that follow may be optional depending on agiven embodiment without departing from the principles provided in thisdisclosure.

FIG. 1 is a high-level network system for accessing, managing, andvisualizing structured data, according to one embodiment.

FIG. 2A is a functional block diagram of a computing environment foraccessing and visualizing structured data, according to one embodiment.

FIG. 2B is a detailed system diagram of FIG. 2A, according to oneembodiment.

FIG. 3 shows an example interface for visualizing statistical dataderived from one or more signal sources, according to one embodiment.

FIG. 4 shows an example interface for configuring one or more signalsdetected from one or more data sources, according to one embodiment.

FIG. 5 depicts an example interface for configuring one or more detectedsignal for analysis, according to one embodiment.

FIG. 6 shows an example interface for validating one or more detectedsignals, according to one embodiment.

FIG. 7 shows and example interface for transitioning one or moredetected signals from a validation phase to an assessment phase,according to one embodiment.

FIG. 8 shows an example interface for displaying a validated signalundergoing further assessment, according to one embodiment.

FIG. 9 shows an example interface for configuring aspects of a detectedsignal undergoing further assessment, according to one embodiment.

FIG. 10 shows an example interface for further configuring an assessmentphase of a signal, according to one embodiment.

FIG. 11 shows an example interface for displaying a reference document,according to one embodiment.

FIG. 12 shows an example interface for displaying another referencedocument, according to one embodiment.

FIG. 13 shows an example interface for generating a signal report,according to one embodiment.

FIG. 14 shows a signal report generated using information in the exampleinterface of FIG. 12 , according to one embodiment.

FIG. 15 shows an example interface for configuring a detected signalundergoing further regulatory review, according to one embodiment.

FIG. 16 shows an example interface for placing a signal in a continuousstate of surveillance, according to one embodiment.

FIG. 17 shows an example flowchart for executing one or more statisticaloperations on a configured signal, according to one embodiment.

DETAILED DESCRIPTION

The figures and descriptions provided herein may have been simplified toillustrate aspects that are relevant for a clear understanding of theherein described devices, systems, and methods, while eliminating, forthe purpose of clarity, other aspects that may be found in typicalsimilar devices, systems, and methods. Those of ordinary skill mayrecognize that other elements and/or operations may be desirable and/ornecessary to implement the devices, systems, and methods describedherein. But because such elements and operations are well known in theart, and because they do not facilitate a better understanding of thepresent disclosure, a discussion of such elements and operations may notbe provided herein. However, the present disclosure is deemed toinherently include all such elements, variations, and modifications tothe described aspects that would be known to those of ordinary skill inthe art.

The terminology used herein is for the purpose of describing particularexample embodiments only and is not intended to be limiting. Forexample, as used herein, the singular forms “a”, “an” and “the” may beintended to include the plural forms as well, unless the context clearlyindicates otherwise. The terms “comprises,” “comprising,” “including,”and “having,” are inclusive and therefore specify the presence of statedfeatures, integers, steps, operations, elements, and/or components, butdo not preclude the presence or addition of one or more other features,integers, steps, operations, elements, components, and/or groupsthereof. The method steps, processes, and operations described hereinare not to be construed as necessarily requiring their performance inthe particular order discussed or illustrated, unless specificallyidentified as an order of performance. It is also to be understood thatadditional or alternative steps may be employed.

Although the terms first, second, third, etc., may be used herein todescribe various elements, components, regions, layers and/or sections,these elements, components, regions, layers and/or sections should notbe limited by these terms. These terms may be only used to distinguishone element, component, region, layer or section from another element,component, region, layer or section. That is, terms such as “first,”“second,” and other numerical terms, when used herein, do not imply asequence or order unless clearly indicated by the context.

System Environment

Illustrated in FIG. 1 is a high level network system 100 for generatinginterfaces that access, maintain, analyze, and display structured data.In the depicted implementation, the system 100 may include a server 105communicatively coupled to a plurality of secure computing environments125 via the network 110. The secure computing environments 125 may inturn be communicatively coupled to a data source 130. While a server 105is illustrated, the disclosed principles and techniques could beexpanded to include multiple servers 105.

The server 105, according to some embodiments, is configured to store aplurality of structured data in a secure environment requiringauthentication before access is granted to the structured data.According to one implementation, the structured data includeshierarchical data having varying and connected categories/levels thatdescribe a plurality of aspects of the structured data. In some cases,the structured data in the server 105 is sourced or obtained fromthird-party scientific sources, and/or from third-party regulatoryagencies, and/or from academic sources, and/or from industrial sources,etc. According to one implementation, the structured data includesterminology data associated with Medical Dictionary for RegulatoryActivities (“MedDRA”) dictionaries, WHODrug dictionary, or the like. Inaddition, the server 105 may be configured to manage or otherwisemaintain the integrity and/or version updates of the structured data sothat a user (e.g., a user of the secure computing environment) does nothave to deal with such data maintenance processes as the structured datachanges and/or grows. In one embodiment, the server 105 provides (e.g.,using a snapshot reference) the most current version of the structureddata to a user of the system. In other embodiments, the server 105 canalso provide historical versions of the structured data when necessaryor needed. Furthermore, the server 105 may include mechanisms thatexecute operations of data decompression operations, data decryptionoperations, and data decoding operations associated with the structureddata so that the user is effectively isolated from such operations.

Moreover, the server 105 allows for easy associating, tagging, or codingnew phenomena/events/cases (e.g., in the medical area or any other area)with structured data from the server 105. In some cases, the tagging orcoding is automatically executed by the server 105. In some instances,coding/tagging decisions are snapshotted to the secure computingenvironment and are not affected by updates occurring at the server 105.In one embodiment, the server 105 enables a user to control versioninformation of the structured data for follow-up cases/events. Inaddition, the server 105 allows a user (e.g., user of the securecomputing environment 125) to transition from accessing structured datafrom the data source 130 to accessing data from the server 105. In oneembodiment, the data accessed from the server 105 and/or from the datasource 130 may include one or more signals associated with a specificproduct. These signals may include statistical data associated with aproduct, testing data associated with a product, regulatory dataassociated with a product, usage data associated with the product, etc.In one embodiment, the product includes a pharmaceutical product, amedical product, or a health-related product.

The server 105 may be implemented within a computing device such as amainframe server, a content server, a communication server, a laptopcomputer, a desktop computer, a handheld computing device, a virtualmachine, a cloud-based computing solution and/or service, and/or thelike. The server 105 may include a plurality of computing devicesconfigured to communicate with one another and/or implement thetechniques described herein. In some instances, the server 105 mayinclude various elements of a computing environment as described withreference to FIGS. 2A and/or 2B. For example, the server 105 may includea processing system 202, a memory 204, an input/output (I/O) system 206,and a communication system 208. A user (e.g., database administrator)may operate/maintain the server 105 either locally or remotely as thecase may require.

The server 105 may be configured to have storage logic that isexecutable to store structured data that is shared across multiplesecure computing environments 125. According to one implementation, theserver 105 includes a plurality of non-volatile/non-transitory storagemedia such as solid state storage media, hard disk storage media,virtual storage media, cloud-based storage drives, storage servers,and/or the like. The plurality of storage media may be configured tostore data from a plurality of sources. For example, the server 105 mayinclude storage logic that is executable to store structured dataderived from, for example, medical data, research data, education data,pharmaceutical data, product data, government data, etc. According tosome implementations, the storage logic of the server 105 may beconfigured to automatically monitor and/or update relevant structureddata obtained from a third-party source. For example, the storage logicof the server 105 may periodically monitor updates associated withstructured data (e.g., dictionary of medical terms for research andregulatory purposes) from third-party organizations/sources andautomatically update different versions of the structured data withinone or more storage media of the server 105. In one embodiment, thestorage logic of the server 105 manipulates or otherwise formats thestructured data such that user interfaces generated by a securecomputing environment 125 can seamlessly access/retrieve and present thestructured data to a user. In addition, structured data from the server105 may be accessed on a regulated basis via credential access, forexample. This regulated basis may be determined, in part, by licenses,privileges, and other levels of authorization dictated by a user'scredentials.

The data store 130 may include storage logic for storing data fromthird-party sources. The data store 130 may include a plurality ofnon-volatile/non-transitory storage media such as solid state storagemedia, hard disk storage media, virtual storage media, cloud-basedstorage drives, storage servers, and/or the like. According to someembodiments, the data store 130 may include logic that updates datastored within its storage devices based on updates issued by the server105. In one embodiment, the data store 130 and/or the server 105 mayinclude one or more point-in-time architecture (PTA) databases.

As previously discussed, the network 110 facilitates communicationbetween the server 105 and the secure computing environment 125. Thenetwork 110 may also allow different secure computing environments 125to communicate with each other. According to one embodiment, the network110 may include a plurality of networks. For instance, the network 110may include any wired/wireless communication network that facilitatescommunication between the components of the network system 100. Thenetwork 110, in some instances, may include an Ethernet network, acellular network, a computer network, the Internet, a wireless fidelity(Wi-Fi) network, a light fidelity (Li-Fi) network, a Bluetooth network,a radio frequency identification (RFID) network, a near-fieldcommunication (NFC) network, a fiber optics network, a laser-basednetwork, and/or the like.

The secure computing environment 125 is configured to generate one ormore user interfaces for accessing, analyzing, and displaying thestructured data. The secure computing environment 125 may also beconfigured to generate and or display interfaces that visualize one ormore signals derived from one or more sources. As discussed furtherbelow, the interfaces generated by the secure computing environment 125may include visualizations of statistical data of a product and mayallow a user to configure different aspects of the statistical data asneeded. According to some implementations, the secure computingenvironment 125 includes functionalities and/or enhanced securityfeatures that allow a user to securely access and/or securely managestructured data and/or manage one or more detected signals. As shownmore clearly in the exemplary functional and system diagrams of FIGS. 2Aand 2B, the secure computing environment 125 includes a processingsystem 202, a memory 204, an I/O system 206, and a communication system208. The processing system 202, the memory 204, the I/O system 206, andthe communication system 208 may include one or more subsystems thatperform one or more of the operations described herein. Additionally,each system of the secure computing environment 125 may be operativelyand/or otherwise communicatively coupled with each other so as tofacilitate one or more operations described herein. The secure computingenvironment 125 may include general hardware, specifically-purposedhardware, and/or a combination thereof.

The processing system 202 may control the memory 204, the I/O system206, and the communication system 208, as well as any includedsubsystems, elements, components, devices, and/or functions performed bythe memory 204, I/O system 206, and the communication system 208.Additionally, any actions described in this disclosure as beingperformed by a processor or one or more processors of a computing deviceor one or more computing device processors and/or one or more computingsystem processors may be executed by the processing system 202 of FIGS.2A and 2B. Further, while one processing system 202 is shown in FIGS. 2Aand 2B, multiple processing systems may be present and/or otherwiseincluded in the secure computing environment 125 or elsewhere in theoverall network system 100 of FIG. 1 . Thus, while instructions may bedescribed as being executed by the processing system 202 (and/or varioussubsystems of the processing system 202), the instructions may beexecuted simultaneously, serially, and/or otherwise by one or multipleprocessing systems 202 on one or more computing devices.

According to one embodiment, the processing system 202 may beimplemented as one or more computer processor chips and/or graphicalprocessing unit (GPU) chips and may include a hardware device capable ofexecuting computer instructions. The processing system 202 may executeinstructions, codes, computer programs, and/or scripts. Theinstructions, codes, computer programs, and/or scripts may be receivedfrom the I/O system 206, the communication system 208, and/or stored inthe memory 204, and/or received from the other subsystems of the securecomputing environment 125 and/or received from other computingenvironments.

In some embodiments, the processing system 202 may include subsystemssuch as a content management subsystem 212, a graphical processingsubsystem 216, and a resource allocation subsystem 218. Each of theaforementioned subsystems of the processing system 202 may becommunicatively or operably coupled to each other.

The content management sub-system 212 may facilitate generation,modification, analysis, transmission, and/or presentation of content.Content may be file content, media content, structured data content,user interfaces, or any combination thereof. In some instances, contenton which the content management system 212 operates includes structureddata from the server 105, structured data from the local repository 130,user interface data, device information, images, text, themes, audiofiles, video files, documents, and/or the like. Additionally, thecontent management subsystem 212 may control the audio-visualenvironment and/or appearance of application data during execution ofvarious processes. In some embodiments, the content management subsystem212 may interface with a third-party content server and/or third-partymemory locations for execution of its operations.

The graphical processing subsystem 216 may facilitate generation,modification, analysis, processing, transmission, and/or presentation ofthe content described above, as well as any data described herein. Insome embodiments, the graphical processing subsystem 216 may be used torender content for presentation on a computing device (e.g., via agraphical user interface of the computing device). The graphicalprocessing subsystem 216 may also include multiple graphical processingsubsystems and therefore may be configured to perform and/or executemultiple processes in parallel. In some implementations, the graphicalprocessing subsystem 216 may be used in conjunction with components ofthe memory 204, the I/O system 206, the communication system 208, and/ora combination thereof.

The resource allocation subsystem 218 may facilitate the determination,monitoring, analysis, and/or allocation of computing resourcesthroughout the secure computing environment 125 and/or other computingenvironments. Computing resources of the secure computing environment125 may be used by the processing system 202, the memory 204, the I/Osystem 206, and/or the communication system 208. These resources mayinclude processing power, data storage space, network bandwidth, and/orthe like. Accordingly, the resource allocation subsystem 218 may includesensors and/or other specially-purposed hardware for monitoringperformance of each system and/or subsystem of the secure computingenvironment 125, as well as hardware for responding to thecomputing-resource needs of each system and/or subsystem. In someembodiments, the resource allocation subsystem 218 may use computingresources of a second secure computing environment separate and distinctfrom the secure computing environment 125 to facilitate a desiredoperation.

The memory 204 may be used for storing, recalling, receiving,transmitting, and/or accessing various files and/or data (e.g.,structured data) and/or signals associated with a product during theoperation of the secure computing environment 125. For example, thememory 204 may store, recall, and/or update structured data and/or oneor more signals from the server 105 and/or from the local repository asthe case may be. In some embodiments, the memory 204 may storeinstructions and/or data that may be executed by the processing system202. For instance, the memory 204 may store instructions that executeoperations associated with one or more systems and/or one or moresubsystems of the secure computing environment 125. For example, thememory 204 may store instructions for the processing system 202, the I/Osystem 206, the communication system 208, and for itself.

Memory 204 may include various types of data storage media such as solidstate storage media, hard disk storage media, virtual storage media,and/or the like. Memory 204 may include dedicated hardware elements suchas hard drives and/or servers, as well as software elements such ascloud-based storage drives. In some implementations, memory 204 may be arandom access memory (RAM) device, a dynamic random access memory (DRAM)device, a static random access memory (SRAM) device, flash memory, readonly memory (ROM) device, and/or various forms of secondary storage. TheRAM device may be used to store volatile data and/or to storeinstructions that may be executed by the processing system 202. Forexample, the instructions stored may be a command, a current operatingstate of secure computing environment 125, an intended operating stateof secure computing environment 125, and/or the like. As a furtherexample, data stored in the memory 204 may include instructions relatedto various methods and/or functionalities described herein. The ROMdevice may be a non-volatile memory device that may have a smallermemory capacity than the memory capacity of a secondary storage of thesecure computing environment. The ROM device may be used to storeinstructions and/or data that may be read during execution of computerinstructions. In some embodiments, access to both the RAM device and ROMdevice may be faster to access than access to the secondary storage ofthe secure computing environment 125. Secondary storage may comprise oneor more disk drives and/or tape drives which may be used fornon-volatile/non-transitory storage of data or as an over-flow datastorage device of the secure computing environment 125 if the RAM deviceis not large enough to hold all working data. Secondary storage may beused to store programs that may be loaded into the RAM device when suchprograms are selected for execution.

Turning back to FIG. 2A, the memory 204 may include subsystems such asapplication data 228, application programming interface 230, contentstorage 232, and cache storage 240. Application data 228 may facilitatedeployment, storage, access, execution, and/or utilization of anapplication utilized by the secure computing environment 125 and/or anyother computing environments described herein. As such, application data228 may store any information and/or data associated with anapplication. Application data 228 may further store various pieces ofinformation and/or data associated with the operation of an applicationand/or with the secure computing environment 125 as a whole, such as astatus of computing resources (e.g., processing power, memoryavailability, resource utilization, and/or the like), runtimeinformation, user interfaces, systems to direct execution of operationsdescribed herein to, user permissions, security credentials, and/or thelike.

The application programming interface (API) 230 may facilitatedeployment, storage, access, execution, and/or utilization ofinformation associated with APIs of secure computing environment 125and/or any other computing environment described herein. For example,secure computing environment 125 may include one or more APIs forvarious devices, applications, systems, subsystems, elements, and/orother computing environments to allow communication between one or moreapplications associated with the secure computing environment 125.Accordingly, API 230 may include API databases containing informationthat may be accessed and/or used by applications, systems, subsystems,elements, and/or operating systems of other devices and/or computingenvironments in communication with the secure computing environment 125.In some cases, the API 230 may enable the server 105 and the securecomputing environment 125 to communicate with each other.

The content storage 232 may facilitate deployment, storage, access,and/or utilization of information associated with structured data asfurther discussed below. In one embodiment, content storage 232 maycommunicate with a content management system 212 to receive and/ortransmit content (e.g., structured data, media content, etc.).

The I/O system 206 may include hardware and/or software elements for thesecure computing environment 125 to receive, and/or transmit, and/orpresent information useful for generating one or more interfaces forretrieving and displaying structured data according to some embodimentsof this disclosure. For example, elements of the I/O system 206 may beused to receive input from a user of the secure computing environment125. As described herein, I/O system 206 may include subsystems such asI/O device 242, I/O calibration subsystem 244, and/or driver 246.

The I/O device 242 may facilitate the receipt, transmission, processing,presentation, display, input, and/or output of information as a resultof executed processes described herein. In some embodiments, the I/Odevice 242 may include a plurality of I/O devices. In some embodiments,I/O device 242 may include a variety of elements that enable a user tointerface with secure computing environment 125. For example, I/O device242 may include a keyboard, a touchscreen, a button, a sensor, abiometric scanner, a laser, a microphone, a camera, and/or anotherelement for receiving and/or collecting input from a user. Additionallyand/or alternatively, I/O device 242 may include a display, a screen, asensor, a vibration mechanism, a light emitting diode (LED), a speaker,a radio frequency identification (RFID) scanner, and/or another elementfor presenting and/or otherwise outputting data to a user. In someembodiments, the I/O device 242 may communicate with one or moreelements of processing system 202 and/or memory 204 to executeoperations associated with generating user interfaces for retrieving andvisualizing structured data.

The I/O calibration system 244 may facilitate the calibration of the I/Odevice 242. For example, I/O calibration system 244 may detect and/ordetermine one or more settings of I/O device 242, and then adjust and/ormodify settings so that the I/O device 242 may operate more efficiently.In some embodiments, I/O calibration system 244 may use a driver 246 (ormultiple drivers) to calibrate I/O device 242 as needed. For example,driver 246 may include software that is to be installed by I/Ocalibration system 244 so that an element of secure computingenvironment 125 (or an element of another computing environment) mayrecognize and/or integrate with I/O device 242.

The communication system 208 may facilitate establishment, maintenance,monitoring, and/or termination of communications between the securecomputing environment 125 and other computing environments, third-partyserver systems, and/or the like. Communication system 208 may alsofacilitate internal communications between various elements (e.g.,systems and/or subsystems) of secure computing environment 125. In someembodiments, communication system 208 may include a network protocolsubsystem 248, an API gateway 250, an encryption engine 252, and/or acommunication device 254. These systems and/or subsystems of thecommunication system 208 may be implemented as hardware, software, or acombination thereof.

The network protocol subsystem 248 may facilitate establishment,maintenance, and/or termination of a communication connection for thesecure computing environment 125 via a network (e.g., network 110). Forexample, network protocol subsystem 248 may detect and/or define acommunication protocol required by a particular network and/or networktype. Communication protocols utilized by network protocol subsystem 248may include Wi-Fi protocols, Li-Fi protocols, cellular data networkprotocols, Bluetooth® protocols, internet protocols, WiMAX protocols,Ethernet protocols, power line communication (PLC) protocols, and/or thelike. In some embodiments, facilitation of communication for the securecomputing environment 125 may include transforming and/or translatingdata from a first communication protocol to a second communicationprotocol. In some embodiments, network protocol subsystem 248 maydetermine and/or monitor an amount of data traffic to determine whichnetwork protocol is to be used for establishing a secure communicationconnection, transmitting data, and/or performing retrieval andsubsequent visualization of structured data.

The application programming interface (API) gateway 250 may allow otherdevices and/or computing environments and/or applications external tothe secure computing environment 125 to access the API 230 of the memory204. For example, a computing system may access the API 230 of thesecure computing environment 125 via the API gateway 250. In someembodiments, API gateway 250 may be required to validate usercredentials associated with a user of a computing device (e.g., a deviceexternal to the secure computing environment 125) prior to providingaccess to the API 230 to the user. API gateway 250 may includeinstructions for the secure computing environment 125 and therebycommunicate with external devices and/or between components of thesecure computing environment 125.

The encryption engine 252 may facilitate translation, encryption,encoding, decryption, and/or decoding of information received,transmitted, and/or stored by the secure computing environment 125.Using encryption engine 252, each transmission of data may be encrypted,encoded, and/or translated for security reasons, and any received datamay be encrypted, encoded, and/or translated prior to its processingand/or storage. In some embodiments, encryption engine 252 may generatean encryption key, an encoding key, a translation key, and/or the like,which may be transmitted along with any data content.

The communication device 254 may include a variety of hardware and/orsoftware specifically purposed to facilitate communication for securecomputing environment 125 with external systems and/or devices. In someembodiments, communication device 254 may include one or more radiotransceivers, chips, analog front end (AFE) units, antennas, processingunits, memory, other logic, and/or other components to implementcommunication protocols (wired or wireless) and related functionalityfor facilitating communication for WAS system 125. Additionally and/oralternatively, communication device 254 may include a modem, a modembank, an Ethernet device such as a router or switch, a universal serialbus (USB) interface device, a serial interface, a token ring device, afiber distributed data interface (FDDI) device, a wireless local areanetwork (WLAN) device and/or device component, a radio transceiverdevice such as code division multiple access (CDMA) device, a globalsystem for mobile communications (GSM) radio transceiver device, auniversal mobile telecommunications system (UMTS) radio transceiverdevice, a long term evolution (LTE) radio transceiver device, aworldwide interoperability for microwave access (WiMAX) device, and/oranother device used for communication purposes.

To manage and evaluate data received from data sources, server 105 mayprovide to each computing environment 125 a number of user interfaces.In this context, signals are data from the data sources that haveprocessed using one or more statistical operations. These userinterfaces may allow each computing environment to manage and evaluateone more detected signals based on the data. Moreover, the userinterface may allow the users of each computing environment 125 toperform signal detection analysis using the statistical metricsdescribed in herein including point-in-time statistical metrics. Also,the one or more detected signals may be configured or otherwiseparameterized using a plurality of stages. The stages may include adetection phase, an analysis and prioritization phase, a validationphase, an assessment phase, a risk management phase, and a regulatoryreview phase. This is described in further detail hereinafter.

User Interfaces

FIG. 3 illustrates an example interface 300 that is generated forvisualizing statistical data derived from one or more signal sources.The one or more signal sources may comprise one or more data sources(e.g., server 105 and/or data source 130 of FIG. 1 ) that storestructured data. Moreover, the interface 300 may be generated in asecure computing environment (e.g., secure computing environment 125 ofFIG. 1 ) according to some embodiments. The interface 300 includes adetails section 302, a statistical analysis chart 304, one or morestatistical metrics 306A . . . 306E, and one or more data sources 308associated with at least one detected signal (e.g., a record, a file, orother data associated with a product). In addition, the interface 300may include event terms E1-E5, one or more repository referencesER1-ER4, and one or more data sources (e.g., data source 1 to datasource 5) associated with at least one detected signals. According toone embodiment, the interface 300 may include other features other thanthose shown in FIG. 3 . Furthermore, the interface 300 may be generatedin response to passing one or more detected signals through a number ofprocessing stages or phases as discussed in association with FIGS. 4-15.

The details section 302 of FIG. 3 provides a summary of informationassociated with one or more detected signals. For example, the detailssection 302 may include information such as the name of a product (e.g.,product 1) and an organization that produces the product (e.g., AgencyA). Thus, the illustrated embodiment of interface 300 represents avisualization of one or more detected signals associated with product 1manufactured by agency A. The statistical analysis chart 304 provides astatistical diagram or a graph, or a plot, or a graphical illustrationbased on the one or more statistical metrics 306A . . . 306E. In oneembodiment, the statistical analysis chart 304 also includes a hoversummary 312 providing specific details associated with specific datapoints within the statistical chart 304. In some cases, the statisticalanalysis chart 304 includes a bubble chart depicting multi-dimensions(e.g., 2-dimensions, 3-dimensions, 4-dimensions, etc.) of dataassociated with one or more detected signals. For example, a bubblecomprised in the bubble chart may have a first dimension representing afirst statistical metric, a second dimension representing a secondstatistical metric, and a third dimension representing a magnitude, anamount, or a size of the bubble. In some cases, the bubble chartbeneficially provides relationships and patters between data sets,multiple values (hundreds or thousands of values), values differing bysubstantially significant orders of magnitude, etc., associated with oneor more detected signals. Such identified relationships can helpprioritize areas of focus of the bubble chart. For example, certainareas of focus of the bubble chart can help identify clusters thatindicate optimal project executions and considerations (e.g., cost,value, and risk) associated with the one or more detected signals. Whilea bubble chart is shown in the illustrated embodiment, the chart 304 mayinclude funnel charts, categorical column charts, unit charts, barcharts, stacked charts, multiple line charts, cherry charts, etc.

In some instances, the interface 300 may represent a signal profileproduct having a chart (e.g., statistical analysis chart 304), a grid(e.g., layout of statistical metrics 306A . . . 306E) and an auto-codedstructured data (e.g., Medical Dictionary for Regulatory Activities(“MedDRA”) dictionaries). The auto-coded structured data may includestructured, tiered or stratified data that allows for words or termsthat are similar to low level terms within the strata of the structureddata to be used for automatic updating of terms associated with one ormore signal sources and/or associated with a product associated with thesignal profile product. Such updates may include takingsnapshots/updates of the low level terms in the strata of the structureddata from a repository and automatically updating similar low levelterms associated with the interface 300. According to one embodiment,automatic coding occurs at a record level such that one or more recordswith terms associated with the interface 300 are updated to a specificversion of records in the repository within which is the structureddata. A snapshot reference may be used to automatically update certainrecords with terms associated with the interface 300 without updatingall the records associated with the interface 300. A snapshot referencemay include a reference object including one or more identifiers, one ormore tagging/coding data, and at least one version of information.According to one embodiment, an automatic coding or update operation maybe executed on one or more terms associated with the interface 300 usingthe tagging data. The update operation may include at leastautomatically updating version information associated with a record/termassociated with the interface 300. The update operation may also includeautomatically updating a plurality of records/terms associated with theinterface 300 based on updated records/terms within a repository (e.g.,a MedDRA dictionary or a WHODrug dictionary).

In one embodiment, the one or more detected signals may be configured orotherwise parameterized using a plurality of stages. Such configuringand/or parameterizing of the one or more detected signals may beinitiated by activating (e.g., clicking) a button such as the button 310of the interface 300. For example, the plurality of stages used toconfigure the one or more detected signals may include a detectionphase, an analysis and prioritization phase, a validation phase, anassessment phase, a risk management phase, and a regulatory reviewphase. These phases are further discussed in association with FIGS. 4-16.

In some embodiments, the statistical metrics 306A . . . 306E may beexclusively different statistical metrics. In some embodiments, eachstatistical metric 306A . . . 306E may include one or more statisticaloperations. In some embodiments, the one more statistical operations mayinclude Empirical Bayes Geometric Mean (EBGM), Proportional ReportingRatio (PRR), Chi-Square, or the like.

FIG. 4 shows an interface 400 for configuring one or more signalsdetected from one or more data sources. In one embodiment, the one ormore signals are associated with a product (e.g., a pharmaceuticalproduct) that has to go through a plurality of regulatory and testingphases in order to be approved for use. As seen in the figure, theinterface 400 includes a first details section 402, a status section 403(e.g., 403 a and 403 b), a second details section 404A, a signal sourcessection 404B, and a signal analysis section 404C associated with aproduct (e.g., product 2). As seen in the figure, the first detailssection 402 includes summary data associated with the second detailssection 404A, the signal sources section 404B, and the signal analysissection 404C. For example, the first details section 402 may include thenumber of sources from which the one or more signals are obtained, thetype of analysis being conducted on the one or more detected signals,reference documents associated with analysis of the one or more detectedsignals, a workflow timeline indicative of an analysis phase of the oneor more detected signals, etc.

The status section 403 indicates a specific stage/phase and/or a statusassociated with a specific stage of a configuration process for one ormore detected signals. For example, the status section 403 a indicatesthat the one or more signals associated with a product have beensuccessfully detected. Successful signal detection may be furtherindicated by activating a first color (e.g., green) associated with thestatus section 403 a. Similarly, the status section 403 b indicates thatan analysis and prioritizing stage of a signal configuration process isongoing and once completed, a second color (e.g., green) associated withthe status section 403 b may be activated to indicate completion of theanalysis and prioritizing stage of the signal.

The second details section 404A provides contextual data associated witha product for which the one or more signals have been detected. Forexample, the second details section 404A may include a first propertyassociated with a product (e.g., blood magnesium level associated withproduct 2), a repository term (e.g., PT term 1), an event identifier(e.g., RAE1) associated with the aforementioned structured data, asignal product profile for the product in question, etc. Moreover, thesignal sources section 404B may include a grid format that outlines oneor more event terms E1 . . . E3 associated with structured data, orassociated with one or more data sources (data source 1 to data source3) from which the one or more detected signals were obtained. The gridformat may also include one or more outcomes 408A (e.g., worst outcomesWO1 . . . WO3) resulting from using the product being analyzed, and mayalso include other signal properties 408B, 408C, 408D, and 408D asneeded. The signal properties (e.g., P1 . . . P4) may include data suchas deaths reported associated with using the product being analyzed,life threatening reactions associated with using the product,hospitalizations associated with using the product, and other seriouscases associated with using the product. Also included in the gridformat are one or more statistical metrics 406A . . . 406D withcorresponding entries within the grid format. A button 412 of the signalsources section 404B may allow for additional analysis features to beadded to the signal sources section 404B. The signal analysis section404C may allow a user to provide parameters that assign degrees ofurgency based on one or more data (e.g., statistical metrics, signalproperties, outcomes, etc.) to thereby decide or otherwise prioritize anorder for managing or handling identified issues (e.g., problems)associated with the product being analyzed. According to one embodiment,activating a button 410 of the interface 400 allows a user to furtherconfigure data associated with the signal analysis section 404C asfurther discussed below in association with FIG. 5 .

FIG. 5 shows an example interface for prioritizing, confirming, orotherwise executing a triage operation on one or more detected signalassociated with a given product (e.g., product 2). In one embodiment, apop-up menu 502 may be generated for executing the triage operation inresponse to a user activating the button 410 of FIG. 4 . The pop-up menu502 includes a first field 502A for selecting an option associated witha seriousness property resulting from using the product, a second field502B for selecting an option associated with a reaction (e.g., anadverse reaction) resulting from using the given product, a third field502C for flagging whether the given product should be on a watch-list, afourth field 502D for selecting an outcome associated with using thegiven product, and at least one checkbox 503 for establishing a verdictassociated with using the given drug. In response to the user executingthe triage operation by selecting or parameterizing aspects of the givenproduct based on fields 502A . . . 502D and based on the at least onecheckbox 503, the pop-up menu 502 may transition the user interface 400into the interface shown in FIG. 6 .

In FIG. 6 , the interface 400 now includes a status section 403 c thatis indicative of a validation stage of processing the one or morereceived signals. At this processing stage, the interface 400 candisplay an evaluation section 404D which includes reaction (e.g.,adverse reaction) data associated with using the given product, exposuredata associated with one or more groups using the given product,population impact data, etc. At the validation stage, the user mayactivate a button 602 to generate the pop-up menu 702 of FIG. 7 . Thepop-up menu 702 may include a plurality of fields 702A . . . 702G thatcan be used to parameterize the one or more detected signals during thevalidation stage. In response to validating the one or more signalsusing the pop-up menu 702, the user may activate the button 704 toupdate the interface 400 to transition to an assessment stage as shownin FIG. 8 .

FIG. 8 shows a transition of the interface 400 from a validation stageto an assessment stage. At this stage of processing the one or moredetected signals associated with a given product (e.g., product 2),information about the one or more signals is compared against one ormore reference documents such as the reference documents 1102 of FIG.11, 1202 of FIG. 12 , and signal report 1400 of FIG. 14 . This ensuresthat any inconsistencies associated with the prior processing stages maybe caught and addressed.

FIG. 9 shows an interface 400 for configuring aspects of a detectedsignal undergoing further assessment, according to one embodiment. Theinterface 400 may be at the assessment stage 403 d. In such a case,clinical relevance of the detected signal may be evaluated using aworkflow task window 902, which provides a listing of workflow tasks 902a, 902 b, 902 c, and 902 d to be addressed by a user. The workflow task902 a may be associated with prevention and reversibility optionsconcerning Product 2. The workflow task 902 b may be associated with thedifferent possibilities for prevention concerning Product 2. Theworkflow task 902 c may be associated with signal severity optionsconcerning Product 2. The listing 902 may be associated with reasons forthe signal severity concerning Product 2. After the user is completedaddressing tasks 902 a, 902 b, 902 c, and 902 d, the user may clickbutton 906 to close workflow task window 902. Moreover, the user maycancel their entries for tasks 902 a, 902 b, 902 c, and 902 d byclicking button 908. A new workflow task window 902 may be provided sothe user can re-enter entries for tasks 902 a, 902 b, 902 c, and 902 d.Once the user is done with workflow task window 902, the user canconfirm that he/she has completed a workflow task portion of theassessment by clicking button 904.

In some embodiments, each workflow task 902 a, 902 b, 902 c, and 902 dis provided with a drop menu of options to select. In some embodiments,the user can write responses in a text box to each workflow task 902 a,902 b, 902 c, and 902 d.

FIG. 10 shows an interface 400 for further configuring an assessmentphase of a signal, according to one embodiment. The interface 400 may beat the assessment stage 403 d. After completing workflow tasks 902 a,902 b, 902 c, and 902 d, a causality determination window 1002 may beprovided to determine if Product 2 is likely the cause of an adverseevent. The causality determination window 1002 may include a number ofcausality determination tasks 1002 a, 1002 b, and 1002 c a user mustanswer. The workflow task 1002 a may be associated with assessing thecausality of an adverse event involving Product 2. The workflow task1002 b may be associated with the reasons for the causality involvingProduct 2. The workflow task 1002 c may be associated with the verdicton causality associated with Product 2. The listing 902 may beassociated with reasons for the signal severity involving Product 2.After the user has completed addressing tasks 1002 a, 1002 b, and 1002c, the user may activate (e.g., click) button 1006 to close workflowtask window 1002. Moreover, the user may cancel entries for tasks 1002a, 1002 b, and 1002 c by activating (e.g., clicking) button 1008. A newworkflow task window 1002 may be provided so the user can re-enterentries for tasks 1002 a, 1002 b, and 1002 c. Once the user is done withworkflow task window 1002, the user can confirm completing the workflowtask portion of the assessment by activating (e.g., clicking) button1004.

In some embodiments, each causality determination tasks 1002 a, 1002 b,and 1002 c may be provided with a drop menu of options to select. Insome embodiments, the user may write in a text box responses to eachcausality determination tasks 1002 a, 1002 b, and 1002 c.

FIG. 11 shows an example interface for displaying a reference document1104, according to one embodiment. The interface 400 may still indicatea status indicator 403 d designating that the detected signal of Product2 is at the assessment stage. In this case, a document window 1102 maybe presented once all information in the workflow task window 902 hasbeen entered. The document window 1102 includes an annotated reference1104 and an information section 1106. The annotated reference 1104 maybe a reference closely aligned with information in workflow task window902 of Product 2, and may be annotated using information provided inworkflow task window 902. The information section 1106 may describegeneral information regarding annotated reference 1104, such as the nameof the reference, title, subtype, document number, who created ormodified the document, version data of the document, and the like.

In some embodiments, the underlying document associated with theannotated reference 1104 may be retrieved for a local or remote documentstorage memory system.

FIG. 12 shows an example interface for displaying a reference document1104, according to one embodiment. The interface 400 may indicate astatus indicator 403 e designating the detected signal of Product 2 isat the risk management stage 403 e. In this case, a document window 1202may be presented once all information in causality determination window1002 has been entered. The document window 1202 may include an annotatedreference 1204 and an information section 1206. The annotated reference1204 may be a reference closely aligned with information in thecausality determination window 1002 of Product 2, and may be annotatedusing information provided in the causality determination window 1002.The information section 1206 may describe general information regardingannotated reference 1204, such as the name of the reference, title,status, lifecycle, document number, who created or modified thedocument, version data of the document, aggregate report, and the like.Under the risk management stage 403 e, reviewing the one or morereference documents, including information regarding the one or moredetected signals, allows one to devise a risk management strategyregarding the producer under review.

In some embodiments, the underlying document associated with theannotated reference 1204 may be retrieved from a local or remotedocument storage memory system.

FIG. 13 shows an example interface for generating a signal report. Asignal report may be generated to notify authorities/regulatorsregarding an adverse event associated with a signal associated withProduct 2, for example. This may be determined by using severalstatistical analyses to detect adverse events associated with usingProduct 2. A signal report generation window 1302 may be provided wheninterface 400 is at the regulatory review stage 403 f. The signal reportgeneration window 1302 may include a first portion 1304, a secondportion 1306, and a third portion 1308.

The first portion 1302 may include profile information retrieved fromvarious stages 403 a . . . 403 d of interface 400 regarding the signalof Product 2 as discussed in FIGS. 3-12 . The second portion 1306 mayfocus on signals of disproportionate reporting. The second portion 1306may list the internal names D1-D3 of the data sources Data Source 1-DataSource 3 used for signal detection, signal priority, total number ofcases associated with each local data source Data Source 1-Data Source3, and statistical analysis information St1-St5 used for signaldetection. Note that each data source may produce a different signalthat is assessed using statistical analysis information St1-St5. Thethird portion 1308 may focus on triage and prioritization of Product 2.The information shown in the third portion 1308 may be retrieved frompop-menu 502 of FIG. 5 .

Once all information is appropriately entered and evaluated, a signalreport 1400 may be generated, as shown in FIG. 14 . The signal report1400 may incorporate information displayed in the signal reportgeneration window 1302. Moreover, the signal report 1400 may be providedto select authorities for determining if Product 2 is viable.

According to one embodiment, FIG. 15 shows an example interface forconfiguring a detected signal undergoing further regulatory review.After a signal report 1400 is generated, a final assessment verdictwindow 1502 may be provided. The final assessment verdict of Product 2may be provided when interface 400 is under regulatory review stage 403f. The final assessment verdict window 1502 may be used to give a finalassessment verdict based on the signals of Product 2, as shown in FIG.13 . In particular, the final assessment verdict window 1502 may includea signal verdict option 1502 a and a tracking status option 1502 b. Thesignal verdict option 1502 a may involve a user selecting severaloptions from a pull-down menu, signifying a signal verdict. The trackingstatus option 1502 b may include a user choosing several options from apull-down menu, indicating a tracking status.

After entering the information for signal verdict option 1502 a andtracking status option 1502 b, the user may click button 1506 to closefinal assessment verdict window 1502. Moreover, the user may cancel theentries 1502 a and 1502 b by activating (e.g., clicking) button 1508.The interface 400 may provide a new final assessment verdict window1502, so the user can re-enter entries for signal verdict option 1502 aand tracking status option 1502 b. Once the user is done with finalassessment verdict window 1502, the user can confirm completing thefinal assessment verdict portion of the Regulatory Review by activating(e.g., clicking) button 1504.

In some implementations, the signal verdict option 1502 a and thetracking status option 1502 b may be manually entered by the user.

FIG. 16 shows an example interface for placing a signal in a continuousstate of surveillance. After completing the final assessment verdict,interface 400 may enter an end stage 403 g. At this stage, the signal ofProduct 2, for example, may be placed on an “ongoing surveillance”protocol 1602 based on the selection of tracking status 1502 b, as shownin FIG. 15 . In this case, tracking status 1502 b may be selected toinitiate an “ongoing surveillance” protocol. The protocol for “ongoingsurveillance” 1600 may be a continuous state of surveillance of Product2 at the end stage 403 g.

In other embodiments, other protocols may be used for managing ormaintaining a different surveillance relationship with Product 2.

Workflow

FIG. 17 shows a flowchart for using a point-in-time architecture (PTA)database. In block 1702, first data from a first data source may bereceived. The first data may include a first record associated with afirst attribute and a first event, and a second record associated with asecond attribute and a second event. In block 1704, the first data in afirst PTA database may be organized using the one or more computingdevice processors. For each record, the first PTA database may include afirst time associated with creation of an entry in the first PTAdatabase for the record and a second time associated with manipulationof the entry in the first PTA database for the record. Access to ahistorical version of the first PTA database without destroying acurrent version of the first PTA database is enabled, using the one ormore computing device processors, as shown in block 1706. In block 1708,the one or more computing device processors may enable access to ahistory of manipulation of the first PTA database.

In some implementations, server 105 of FIG. 1 may implement the firstPTA database, the second PTA database, or more PTA databases. In otherimplementations, server 105 may remotely access the first PTA databaseand the second PTA database using the communication system 208 of FIGS.2A and 2B. In other implementations, the one or more computing deviceprocessors may be one or more of the processors 202 of FIGS. 2A and 2B.

In some implementations, server 105 may organize the first data in thefirst PTA database by modifying the first data from the first datasource to be compatible with the first PTA database. In someimplementations, server 105 may send query commands to the first PTAdatabase or the second PTA database to access the first data.

In some embodiments, server 105 may enable access to the history ofmanipulation by providing a listing of the history of manipulation. Insome embodiments, the listing may be a database listing or text file. Insome embodiments, server 105 utilizes logic that adjust the second datato be compatible with the first PTA database or the second PTA database

In block 1710, the second data from a second data source is receivedusing the one or more computing device processors. The second data mayinclude a third record associated with a third attribute and a thirdevent, and a fourth record associated with a fourth attribute and afourth event. The second data may be organized in the first PTA databaseor a second PTA database, organizing using the one or more computingdevice processors, as shown in block 1712. In block 1714, a firstinstruction may be received to process the first data and the seconddata using a first statistical operation using the one or more computingdevice processors. The first statistical operation may include one ormore first parameters. The first statistical operation for the firstdata and the second data may be executed using the one or more computingdevice processors, as shown in block 1716.

In some implementations, server 105 may organize the first data in thefirst PTA database by modifying the first data from the first datasource to be compatible with the first PTA database. In someimplementations, server 105 may send query commands to the first PTAdatabase or the second PTA database to access the first data. In someimplementations, server 105 may send query commands to the first PTAdatabase or the second PTA database to access the second data.

In some embodiments, the first statistical operation may include thesame statistical operations used to define the statistical metrics 306 a. . . 306 e. In some embodiments, the statistical operation may includeone or more point-in time statistical operations.

In some implementations, server 105 may receive the first instruction toprocess the first data and the second data using the first statisticaloperation. In some implementations, server 105 determines if theinformation in the first data and the second data valid to perform thefirst statistical operation. In some implementations, interface 300allows a user to evaluate the first data and the second data beforeexecuting the first statistical operation.

In block 1718, a second instruction may be received to process the firstdata and the second data using a second statistical operation, whereinthe second statistical operation comprises one or more second parametersusing the one or more computing device processors. The secondstatistical operation may be executed for the first data and the seconddata using the one or more computing device processors, as shown inblock 1720. In block 1722, a third instruction may be received to filterfirst output data resulting from the execution of the first statisticaloperation for the first data and the second data, or filter secondoutput data resulting from the execution of the second statisticaloperation for the first data and the second data using the one or morecomputing device processors. The filtering of the first output data orthe second output data is based on a user-selected attribute associatedwith records comprised in the first data or the second data,respectively. In block 1724, simultaneous graphical display of the firstoutput data resulting from the execution of the first statisticaloperation for the first data and the second data, and graphical displayof the second output data resulting from the execution of the secondstatistical operation for the first data and the second data may beenabled, on a single user interface. Note the first output data and thesecond output data may be considered signals because they have processedusing one or more statistical operation.

In some embodiments, the first statistical operation may include thesame statistical operations used to define the statistical metrics 306A. . . 306E. In some embodiments, the statistical operation may includeone or more point-in time statistical operations. In some embodiments,interface 300 via server 105 may be used define which one of thestatistical operations. In some implementations, interface 300 allows auser to evaluate the first data and the second data before executing thesecond statistical operation. In some embodiments, the first statisticaloperation and the second statistical operation include point-in-timestatistical metrics or operations, such as EBGM or the like.

In some embodiments, the user-selected attribute associated with therecords may include point-in-time statistical metrics, such as EBGM orthe like.

In some embodiments, server 105 receives the second instruction toprocess the first data and the second data using the second statisticaloperation. In some embodiments, server 105 determines if the informationin the first data and the second data can be used to perform the secondstatistical operation. In some embodiments, interface 400 via server 105may be used define which one of the statistical operations is used atthe analysis and prioritization stage 403 b.

In some implementations, server 105 receives the third instruction tofilter first output data resulting from the execution of the firststatistical operation for the first data and the second data, or filtersecond output data resulting from the execution of the secondstatistical operation. In some implementations, server 105 validates thefirst output data and the second output data.

In some implementations, interface 300 may receive the first instructionor the second instruction to process the first data and the second datausing the first statistical operation or the second statisticaloperation. In some implementations, the user interface may allow a userto evaluate the first data and the second data before performing thefirst statistical operation. In some implementations, the interface 300may display the first and the second data before performing the firststatistical operation or the second statistical operation. In someimplementations, the interface 300 may be configured to simultaneouslydisplay the first output data and the second output to one or moreusers. In some implementations, the interface 300 may further includedisplaying information associated with the first output data and thesecond output data.

Once the first output data and the second output are identified,interface 400 of FIGS. 3-16 may identify the first output data and thesecond output data as one or more signals associated with a product.

In some embodiments, interface 400 of FIGS. 3-16 may detect one or moresignals associated with the first output data and the second outputdata. In some embodiments, interface 400 allows for the one or moredetected signals to be parameterized using a plurality of stages. Insome embodiments, the plurality of stages used to configure the one ormore detected signals may include a detection phase, an analysis andprioritization phase, a validation phase, an assessment phase, a riskmanagement phase, and a regulatory review phase, as discussed in FIGS.3-16 . In some embodiments, interface 400 may manage the one or moresignals associated with the first output data and the second output datafor later processing.

Reference in the specification to “one implementation” or “animplementation” means that a particular feature, structure, orcharacteristic described in connection with the implementation isincluded in at least one implementation of the disclosure. Theappearances of the phrase “in one implementation,” “in someimplementations,” “in one instance,” “in some instances,” “in one case,”“in some cases,” “in one embodiment,” or “in some embodiments” invarious places in the specification are not necessarily all referring tothe same implementation or embodiment. In some embodiments, the terms“signal,” “data,” and/or “information” may be used interchangeably. Insome embodiments, signals refer to non-transitory signals.

Finally, the above descriptions of the implementations of the presentdisclosure have been presented for the purposes of illustration anddescription. It is not intended to be exhaustive or to limit the presentdisclosure to the precise form disclosed. Many modifications andvariations are possible in light of the above teaching. It is intendedthat the scope of the present disclosure be limited not by this detaileddescription, but rather by the claims of this application. As will beunderstood by those familiar with the art, the present disclosure may beembodied in other specific forms without departing from the spirit oressential characteristics thereof. Accordingly, the present disclosureis intended to be illustrative, but not limiting, of the scope of thepresent disclosure, which is set forth in the following claims.

What is claimed is:
 1. A method for using a point-in-time architecture(PTA) database, the method comprising: receiving, using one or morecomputing device processors, first data from a first data source,wherein the first data comprises a first record associated with a firstattribute and a first event, and a second record associated with asecond attribute and a second event; entering, using the one or morecomputing device processors, the first data into a first PTA database,wherein for each record, the first PTA database comprises a first timeassociated with creation of an entry in the first PTA database for therecord or a second time associated with manipulation of the entry in thefirst PTA database for the record; receiving, using the one or morecomputing device processors, a first instruction to process the firstdata using a first statistical operation, wherein the first statisticaloperation comprises one or more first parameters; executing, using theone or more computing device processors, the first statistical operationfor the first data; after executing the first statistical operation forthe first data, receiving, using the one or more computing deviceprocessors, a second instruction to filter first output data resultingfrom the execution of the first statistical operation for the firstdata, wherein the filtering of the first output data is based on auser-selected attribute associated with records comprised in the firstdata; enabling, using the one or more computing device processors,graphical display of the first output data resulting from the executionof the first statistical operation for the first data, and firstinformation associated with the first output data; enabling, using theone or more computing device processors, graphical display of a firststage of a data processing operation for the first output data;receiving, using the one or more computing device processors, first userinput for the first stage of the data processing operation for the firstoutput data, wherein the first user input comprises at least one firstconfiguration parameter for executing the first stage of the dataprocessing operation; executing, using the one or more computing deviceprocessors, based on the at least one first configuration parameter, thefirst stage of the data processing operation for the first output data;after executing the first stage of the data processing operation for thefirst output data, enabling, using the one or more computing deviceprocessors, graphical display of a second stage of the data processingoperation for the first output data; receiving, using the one or morecomputing device processors, second user input for the second stage ofthe data processing operation for the first output data, wherein thesecond user input comprises at least one second configuration parameterfor executing the second stage of the data processing operation for thefirst output data; executing, using the one or more computing deviceprocessors, based on the at least one second configuration parameter,the second stage of the data processing operation for the first outputdata; after executing the second stage of the data processing operationfor the first output data, enabling, using the one or more computingdevice processors, graphical display of a first option to return to thefirst stage of the data processing operation for the first output dataor a second option to reconfigure the at least one first configurationparameter or the at least one second configuration parameter; andproviding a user interface for displaying a plurality of stages formanaging information in the first output data, wherein the plurality ofstages comprises: a detection stage for detecting the first output data;an analysis or prioritization stage for analyzing or prioritizing thefirst output data based on the information in the first output data; avalidating stage for validating the first output data; and an assessmentstage for comparing the information in the first output data to one ormore reference data.
 2. The method of claim 1, wherein receiving thefirst data into the first PTA database and comprises modifying the firstdata to be compatible with the first PTA database.
 3. The method ofclaim 1, further comprising sending query commands to the first PTAdatabase.
 4. The method of claim 1, wherein the detection stage enablesdisplaying the first output data.
 5. The method of claim 1, wherein theone or more reference data comprise one or more reference documentsassociated with the information in the first output data, and whereinthe information in the first output data is compared to the one or morereference documents.
 6. The method of claim 1, wherein the plurality ofstages further comprises a regulatory review stage, wherein theregulatory review stage produces a data report regarding the firstoutput data based on the regulatory information.
 7. A system for using apoint-in-time architecture (PTA) database, the system comprising atleast one PTA database; one or more computing device processors; and oneor more computing device memories, coupled to the at least one PTAdatabase and the one or more computing device processors, the one ormore computing device memories storing instructions executed by the oneor more computing device processors, wherein the instructions areconfigured to: receive first data from a first data source, wherein thefirst data comprises a first record associated with a first attributeand a first event, and a second record associated with a secondattribute and a second event; enter the first data into a first PTAdatabase, wherein for each record, the first PTA database comprises afirst time associated with creation of an entry in the first PTAdatabase for the record or a second time associated with manipulation ofthe entry in the first PTA database for the record; receive a firstinstruction to process the first data using a first statisticaloperation, wherein the first statistical operation comprises one or morefirst parameters; execute the first statistical operation for the firstdata; after executing the first statistical operation for the firstdata, receive a second instruction to filter first output data resultingfrom the execution of the first statistical operation for the firstdata, wherein the filtering of the first output data is based on auser-selected attribute associated with records comprised in the firstdata; enable graphical display of the first output data resulting fromthe execution of the first statistical operation for the first data, andfirst information associated with the first output data; enablegraphical display of a first stage of a data processing operation forthe first output data; receive first user input for the first stage ofthe data processing operation for the first output data, wherein thefirst user input comprises at least one first configuration parameterfor executing the first stage of the data processing operation for thefirst output data; execute, based on the at least one firstconfiguration parameter, the first stage of the data processingoperation for the first output data; after executing the first stage ofthe data processing operation for the first output data, enablegraphical display of a second stage of the data processing operation forthe first output data; receive second user input for the second stage ofthe data processing operation for the first output data, wherein thesecond user input comprises at least one second configuration parameterfor executing the second stage of the data processing operation for thefirst output data; execute, based on the at least one secondconfiguration parameter, the second stage of the data processingoperation for the first output data; after executing the second stage ofthe data processing operation for the first output data, enablegraphical display of a first option to return to the first stage of thedata processing operation for the first output data or a second optionto reconfigure the at least one first configuration parameter or the atleast one second configuration parameter; and provide a user interfacefor displaying a plurality of stages for managing information in thefirst output data, wherein the plurality of stages comprises: adetection stage for detecting the first output data; an analysis orprioritization stage for analyzing or prioritizing the first output databased on the information in the first output data; a validating stagefor validating the first output data; and an assessment stage forcomparing the information in the first output data to one or morereference data.
 8. The system of claim 7, wherein the instructions arefurther configured to adjust the first data to be compatible with thefirst PTA database.
 9. The system of claim 7, wherein the instructionsare further configured to send query commands to the first PTA database.10. The system of claim 7, wherein the first statistical operationcomprises an Empirical Bayes Geometric Mean (EBGM)-based operation aProportional Reporting Ratio (PRR)-based operation, or aChi-Square-based operation.
 11. The system of claim 7, wherein theinstructions are further configured to provide a second user interfacefor allowing a user to evaluate the first data before executing thefirst statistical operation, wherein the user interface displays thefirst data before executing the first statistical operation.
 12. Thesystem of claim 7, wherein the instructions are further configured toprovide a second user interface for displaying the first output data,wherein the user interface further includes second informationassociated with the first output data.
 13. The system of claim 7,wherein the instructions are configured to further provide a second userinterface for managing the first output data, wherein the user interfaceallows for the first output data to be parameterized for processing. 14.The method of claim 1, wherein the first stage of the data processingoperation or the second stage of the data processing operation comprisesone or more of a detection phase, an analysis and prioritization phase,a validation phase, an assessment phase, a risk management phase, and aregulatory review phase.
 15. The system of claim 7, wherein the firststage of the data processing operation or the second stage of the dataprocessing operation comprises one or more of a detection phase, ananalysis and prioritization phase, a validation phase, an assessmentphase, a risk management phase, and a regulatory review phase.
 16. Thesystem of claim 7, wherein the instructions are further configured tomodify the first data to be compatible with the first PTA database.